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Abstract. Modular polynomials are an important tool in many algorithms involv- 
ing elliptic curves. In this article we investigate their generalization to the genus 2 
case following pioneering work by Gaudry and Dupont. We prove various properties 
of these genus 2 modular polynomials and give an improved way to explicitly compute 
them. 



1. Introduction 

The 'classical' modular polynomial $7v ^ y] was introduced by Kronecker 
more than 100 years ago. The polynomial $Ar is a model for the modular curve 
Yo{N) parametrizing cyclic A^-isogenies between elliptic curves. As is shown in the 
examples below, the explicit computation of $Ar has led to various speed ups in 
algorithms using elliptic curves. 

Examples. 1. Schoof's original algorithm [25] to count the number of points on an 
elliptic curve E/Fp was rather impractical as one had to compute with the complete 
/-torsion of E for various small primes / 7^ p. The key to the improvements made by 
Atkin and Elkies [24, Sections 6-8] is to work with a one- dimensional eigenspace of 
E[l] = Z/IZ X Z/IZ. Instead of using division polynomials of degree — l)/2 one 
can now use the modular polynomial $z of degree / + 1. The 'Schoof-Elkies-Atkin'- 
algorithm behaves very well in practice, and primes p of several thousand digits are 
now feasible [13]. 

2. Both the primality proving algorithm ECPP [9, Section 14D] and efficient 
constructions of cryptographically secure elliptic curves [8, Chapter 18] rely on the 
computation of the Hilbert class polynomial Hq for a certain imaginary quadratic 
order O. At the moment, the fastest known algorithm to compute Ho G Z[A] is 
to first compute its reduction modulo various primes p and then apply the Chinese 
remainder theorem [4, 28]. If we know one root of Hq in Fp, then we can compute 
the other roots by exploiting the modular polynomials for small primes / gener- 
ating the class group of O. This observation is crucial to the practical behaviour of 
the 'CRT-algorithm'. 
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The algorithms in the examples above have analogues for genus 2, see [15, 12]. 
Whereas the situation in well understood in genus 1 and algorithms are fast, the 
computations are still in their infancies in genus 2. Typically, algorithms only ter- 
minate in a reasonable amount of time for very small examples. Inspired by the 
speed ups modular polynomials give in the genus 1 case, we investigate modular 
polynomials for genus 2 in this paper. 

In [17], Gaudry and Schost examine a tailor-made variant of $Ar in genus 2 to 
improve point counting on genus 2 curves over finite fields. The polynomial they 
construct has factorization properties similar to $7v, and this enables them to speed 
up point counting in genus 2 in the spirit of the improvements made by Atkin and 
Elkies in the genus 1 case. 

In this paper, we consider a 'direct' generalization of $Ar to genus 2. Such a gen- 
eralization was first defined by Gaudry in his PhD-thesis [14, Ch. 3] . Whereas we 
have one polynomial in the genus 1 case, we now get 3 polynomials P/v, Qn, Rn 
for every > 2. At the time, it was not possible to explicitly compute these poly- 
nomials Pn,Qn, Rn in the simplest case N = 2. Dupont considered the problem 
of computing the modular polynomials in this thesis [10, Sec. 10.4] as well. He was 
able to compute them for A" = 2, and he gives some partial results for A = 3. The 
resulting polynomials are huge. Nevertheless, knowing just a few modular poly- 
nomials will speed up the 'CRT- algorithm' [12] to compute class fields of degree 4 
CM-fields. This in turn will lead to faster algorithms to construct cryptographically 
secure Jacobians of hyperelliptic curves. This paper solely focuses on the definitions 
and properties of modular polynomials for genus 2 however. 

We reconsider the polynomials defined by Gaudry and Dupont in this article. We 
will mostly restrict to the case that N = pis prime. By combining the function field 
approach of Gaudry with finite symplectic geometry over Fp, we are able to prove 
Gaudry's conjecture on the degree of the modular polynomials. Furthermore, we 
prove that the polynomials Pp, Qp, Rp have coefficients in Q(ii, j2, js)- We also give 
a heuristic in Section 6 implying that the bit size of the polynomial Pp grows likep^^. 
We extend Dupont's results in the following way: the algorithm in [10] requires 
'guessing' the denominator of the coefficients of Pp, Qp, Rp. In Section 6 we prove 
that the denominator is closely linked to the Humbert surface of discriminant 
describing (p, p)-split Jacobians. Our result considerably helps the computation. 

The structure of this article is as follows. After recalling some classical facts of 
polarized abelian varieties in Section 2, we consider level structures in Section 3, 
and for primes p we define the 2-dimensional analogue Yq ' (p) of the curve Yq (p) . In 
Section 4 we consider functions on Yq (p) and this leads naturally to the definition 
of the genus 2 modular polynomials. We explain the moduli interpretation behind 
Pp, Qp, Rp, and compute their degrees. By studying the Fourier coefficients of the 
Igusa j-functions, we show in Section 5 that the genus 2 modular polynomials 
have rational coefficients. Section 6 focuses on the computation of the modular 
polynomials. In the special case p = 2, the modular polynomials are closely linked 
to the Richelot isogeny. We explain the precise relation in Section 7. 
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2. Polarized abelian varieties 

We recall some classical facts about complex abelian varieties. For modular poly- 
nomials we are only interested in the 2-dimensional case, but as much of the theory 
generalizes, we work in arbitrary dimension ^ > 1 in this section. 

Let A/C be a (/-dimension abelian variety, and write A'^ — Pic^{A) for its dual. 
It is well known that every abelian variety admits a polarization, i.e., an isogeny 
ip : A ^ A'^ . If (f is an isomorphism, we call A principally polarized. The Jacobian 
of a curve is the classical example of a principally polarized abelian variety: the 
choice of a base point on the curve determines the polarization. 

The complex points A{C) have the natural structure of a gr- dimensional complex 
torus C^/L. Here, L C is a full lattice. In order to characterize those complex 
tori C^/L that arise as abelian varieties, we define the polarization of a torus. 
We follow the classical approach via Riemann forms. A skew-symmetric form E : 
L X L ^ Z can be extended to a form : x ^ R, and we call E a Riemann 
form if 

1. E{x, y) = E{ix, iy) for all x,y e 

2. the Hermitian form H{x, y) = E{ix, y) + iE{x, y) is positive definite. 

A torus /L is called polarizible if it admits a Riemann form. The link with the 
definition from the previous paragraph is that the Hermitian form H from condition 
2 defines a map 

: (CVL) {Cyiy 

sending x to H{x, •). If (pE is an isomorphism, we say that the torus C^/L is prin- 
cipally polarized. The following theorem describes the precise connection between 
complex abelian varieties and polarizable tori. 

Theorem 2.1. The category of com,plex abelian varieties is equivalent to the cat- 
egory of polarizable tori via the functor A ^(C). 

Proof. See [5, Chapter 4]. □ 

Let L be a full lattice in such that the torus C^/L is principally polarizable. 
One can choose a basis of L such that L is given by -|- Z^r with r a complex 
g X ^r-matrix. The fact that the lattice admits a Riemann form now translates into 
the fact that r is an element of the ^-dimensional Siegel upper half plane 

= {r e Matg(C) | = r, Im(T) positive definite}. 

Conversely, for r e the torus C^/(Z^ -|- Z^r) is principally polarizable. The 
Hermitian form given by H{x^ y) = xIm{T)~^y'^ is by construction positive definite, 
and the Riemann form is given by E{x, y) — lm{H{x, y)). 

Next we consider what happens if we change a basis for the 2(7-dimensional 
polarized lattice L. By the 'elementary divisor theorem', we can choose a basis for 
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L such that the Hermitian form H : L x L ^ 7i is given by the matrix 

1, 



J 



( 



where Ig denotes the g x g identity matrix. The general linear group GL{2g, Z) 
stabilizes the lattice L, and the subgroup 

Sp(2^, Z) = {M e GL(2^, Z) I M JM^ = J} C GL(2^, Z) 

that respects the Hermitian form is called the symplectic group. Explicitly, a matrix 

(c d) G GL(2^, Z) is symplectic if and only if the 5rx^ matrices a, 6, c, d satisfy 

the relations ab^ = b^a, cdF = dFc and adF — b(F = Ig. 

For ^ j e Sp(25r, Z) and r e H^, the g x ^r-matrix cr+d is invertible. Indeed, if 

CT + d had determinant zero, there would be an element y e with yIm(T)y-^ = 0, 
contradicting that Im(T) is positive definite. We define an action of the symplectic 
group Sp(25r, Z) on the Siegel upper half plane by 



(::> 



' CT + d ^ ^ 



where dividing by cr + d means multiplying on the right with the multiplicative 
inverse of the g x (/-matrix cr + d. An explicit check shows that the right hand side 
of (2.1) indeed lies in Hp. 
The map 

r^A^ = C»/(Z» + ZV). 

induces a canonical bijection between the quotient space Ag — Sp(2gf, Z)\Hg and 
the set of isomorphism classes of principally polarized ^-dimensional abelian vari- 
eties. In fact, the space Ag is a coarse moduli space for principally polarized abelian 
varieties of dimension g. 

We close this section by zooming in on the 1-dimensional case, i.e., the case of 
elliptic curves. An elliptic curve is isomorphic to its dual, so polarizations do not 
play a real role here. Indeed, every complex torus is polarizable, and the Siegel space 
Hi equals the Poincare upper half plane H. The symplectic group Sp(2, Z) equals 
the special linear group SL2(Z) in this case. The space Sp2(Z)\H is in canonical 
bijection with the set of isomorphism classes of elliptic curves. 

3. ISOGENIES 

Let A/C be a 2-dimensional principally polarized abelian variety, and let A?" > 1 
be a positive integer. The A- torsion ^[A] of A is, non-canonically, isomorphic to 
(Z/AZ)^. The polarization on A induces a symplectic form v on the rank 4 (Z/AZ)- 
module A[N]. We choose a basis for A[N] such that v is given by the matrix 
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and we let Sp(4, X/NT^) be the subgroup of the matrix group GL(4, T^/NX) that 
respects v. A subspace G C A[N] is called isotropic if v restricts to the zero-form 
on G X G, and we say that A and A' are [N, A'")-isogenous if there is an isogeny 
A A' whose kernel is isotropic of order A^^ . 

The full congruence subgroup r*^^''(A^) of level is defined as the kernel of the 
reduction map Sp(4, Z) Sp(4, Z/iVZ). Explicitly, a matrix ("J) is contained 
in r^^)(A^) if and only if we have a,d = I2 mod N and 6, c = O2 mod N. The 
congruence subgroup r(^)(A'") fits in an exact sequence 

1 — > r(2)(iV) — > Sp(4,Z) — > Sp(4,Z/iVZ) — > 1. 

The surjectivity is not completely trivial, see [1, Lemma 3.2]. 

The 2-dimensional analogue of the subgroup ro(A^) C SL2(Z) occuring in the 
equality Yq{N) — ro(A^)\H of Riemann surfaces is the group 

^^o\n) = { ^) e Sp(4, Z) I c = O2 mod Arj . 

Prom now on, we restrict to the case N = p prime. The reason for this restriction 

is that the finite symplectic geometry we need in the remainder of this section is 

much easier for vector spaces over finite fields than for modules over arbitrary finite 

(2) 

rings. The following lemma gives the link between the group Fq (p) and isotropic 
subspaces of the p-torsion. 

(2) 

Lemma 3.1. The index [Sp(4, Z) : Fq (p)] equals the number of 2-dimensional 
isotropic subspaces of the Fp-vector space F^. 

Proof. We map vl^^p) to a subgroup H C Sp(4,Fp). The inclusion T^^\p) C 

r''o\p) shows that we have [Sp(4, Z) : r^o\p)] = [Sp(4, Fp) : H]. The group H is 
parabolic, and occurs as stabilizer of the 2-dimensional isotropic subspace Fp x Fp x 
X of the symplectic space Fp. The group Sp(4, Fp) permutes the 2-dimensional 
isotropic subspaces transitively by Witt's extension theorem [2, Theorem 3.9], and 
the lemma follows. □ 

Let S{p) be the set of equivalence classes of pairs with A a 2-dimensional 

principally polarized abelian variety and G (Z A\p\ a 2-dimensional isotropic sub- 
space. Here, two pairs (A, G) and {A', G') are said to be isomorphic if there exists 
an isomorphism of abelian varieties ip : A ^ A' with <f{G) = G' . 

Theorem 3.2. The quotient space Fq '{p)\Il2 is in canonical bisection with the set 
S{p) via F^^^(p)t ^ (A^,((l/p,0,0,0),(0,l/p,0,0))) where A^ = C^jiT? + T?t) 
is the variety associated to r. 

Proof. The group Fq •'(p) stabilizes the subspace G — 0, 0, 0), (0, 0, 0)) 

of A\p\, and the image (^1^,(7) therefore does not depend on the choice of a repre- 
sentative T. 
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If At- and A^/ are isomorphic, then there exists e Sp(4, Z) with i/j'T = r'. If an 
isomorphism Ar Ar' maps the group G to G' = {{1/p, 0, 0, 0), (0, 0, 0)) C 
^T-'[p], then lies in Fq (p). Hence, our map is injective. 

To prove surjectivity, we first note that every 2-dimensional principally polarized 
abelian variety occurs as some A^ by Theorem 2.1. The theorem now follows directly 
from Lemma 3.1. □ 

As a quotient space, the 2-dimensional analogue of the curve Yq{p) is 

We close this section by showing how to give Yq (p) the structure of a quasi- 
projective variety. Siegel defined a metric on H2, a generalization of the Poincare 
metric in dimension 1, that respects the action of the symplectic group. With this 
metric, Yq '{p) becomes a topological space. Just as in the 1-dimensional case Yo{p), 
it is not compact. There are several ways to compactify it, one of which is the Satake 
compactification 

Yi'\pr = Yi'\p)UYo{p)U'P\Q). 

By the Baily-Borel theorem [3], the space Yq'^\p)* has a natural structure as a 
projective variety V, and the space Yq'^^(p) C F is therefore naturally a quasi- 
projective variety. 

4. Functions 

The left-action of Sp(4, Z) on the Siegel upper half plane H2 induces a natural 
right-action on the set of functions from H2 to P^(C) via (/M)(r) = /(Mr). The 
fixed points under this action are called rational Siegel modular functions. Igusa 
defined [18, Theorem 3] three algebraically independent rational Siegel modular 

fimctions H2 — ^ P^(C) that generate the function field K of A2 = Yq'^\i). The 
functions ji, J2, J3 that most people use nowadays are slightly different from Igusa's 
and we recall their definition first. 

Let Ek{T) — + ^)~'' 2-dimensional Eisenstein series. Here, 

the sum ranges over all co-prime symmetric 2 x 2-integer matrices that are non- 
associated with respect to left-multiplication by GL(2, Z). We define 

Xio = -43867 • 2-12 . 3-5 . 5-2 . 7-1 . 53-i(£;^£;g _ e^q) 

and 

X12 = 131 • 593 • 2-12 • 3"^ • 5"^ • 7"^ • 337"i(32 • 7^EI + 2 • b^E^ - 691£;i2), 

where the constants in xio and Xi2 should be regarded as 'normalization factors'. 
We then define 

O 05X12 • o-3q3-^4Xi2 • 0-5 0-^6X12 ,0-3 q2-^4Xi2 
ji = Z-6 — , J2 = ^ o — 2 — » 33 = ^ -3 3 \- Z -6 — 4 — . 

Xio Xio Xio Xio 
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We have K = C{ji, j2, js) and via the moduh interpretation for A2, the Igusa 
functions are functions on the set of principally polarized 2-dimensional abelian 
varieties. The functions ji,j2,j3 have poles at r e H2 corresponding to products 
of elliptic curves with the product polarization. 
For a fixed prime p, we define three functions 

j,,^:H2^pi(C) (4.1) 

by 3i,pij) ^ jiiP'^)- These functions arise naturally in the study of (p, p)-isogenous 
abelian varieties as we have 

ji,piA^/{{l/p, 0, 0, 0), (0, 1/p, 0, 0))) = V) = kpi^)- 

Lemma 4.1. The functions ji^p defined in (4.-1) above are invariant under the 
action of v']q\p). 

Proof. Let M = (" ^) be an element of V^q\p), and write c = pc' with c' a 
2 X 2-matrix with integer coefficients. We compute 

j,,p(M • r) = i,(p(M • r)) = U{{j)aT + ph) / {cr + d)) = MB-pr), 
with S = ^^). It is clear that B is again symplectic, so we have ji{B ■ pr) = 

The functions ji^p have poles at r e H2 corresponding to {p,p)-split principally 
polarized abelian varieties, i.e., varieties that are (p, p)-isogenous to a product of 
elliptic curves with the product polarization. 

(2) 

Lemma 4.2. For a prime p, the function field of Yq '{p)/C equals K{ji^p) for 
every i = 1,2, 3. 

(2) 

Proof. The function field of ^2 = ^0 (1) equals K = C(ji, j^, Js) and the function 

(2) (2) 
field C(yo (p)) is an extension of K of degree [Sp(4, Z) : Fq (p)]. The functions ji^p 

are contained in C{Yq'^\p)) by Lemma 4.1. It suffices to show that, for fixed i, the 
functions {jiA^'^)} aeT^^\p)\Sp{4 z) distinct. If two of these functions are equal, 
then the stabilizer S C Sp(4, Z) of ji^p inside Sp(4, Z) strictly contains T^\p). The 
images of S and t'q\p) under the reduction map tt : Sp(4, Z) -» Sp(4, Fp) then 
satisfy 

n{S)Dn{T^^\p)). 

The group 7r(rQ^''(p)) is the stabilizer of an isotropic subspace of Sp(4, Fp) and is 
therefore maximal by [19, Theorem 4.2]. Hence, 7r{S) equals the full group Sp(4, Fp) 
and S has to equal Sp(4, Z). This is absurd. □ 
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The pth modular polynomial Pp for ji is defined as the minimal polynomial of 

ji,p over K = C{ji, j2, js)- It has degree [Sp(4, Z) : Tq ' (p)] and its coefficients 
are rational functions in ji,j2,j3 with complex coefficients. The evaluation map 
(fr ■ C(ji,j2, js) C sending ji to ji(r) maps Pp to a polynomial Pp^r e C[X]. 
The roots of Pp^r are the ji-invariants of principally polarized abelian varieties that 
are (p, p)-isogenous to a variety with j-invariants ii(T), J2(t), J3(t). 

The functions j2,p, js,p are contained in K{ji^p) = K[ji^p] and we define Rp, Qp e 
C{ji,j2,j3)[X] to be the monic polynomials of degree less than deg(Pp) satisfying 

j2,p = Rp{h,p) h,p = QpUhp)- (4.2) 

The evaluation map maps Qp, Rp to polynomials Qp^r, Rp,T G If a; e C 

is a root of -Pp,r) then 

are j-invariants of a principally polarized abelian variety that is (p, p)-isogenous to 
a variety with invariants ii(T), J2(t), J3(t). 

5. Field of definition 

A holomorphic map ifj : H2 ^ C is called a Siegel modular form of degree > if 
it satisfies the functional equation 

for all matrices ^) G Sp(4, Z). The Eisenstein series are Siegel modular 
forms of degree w. Any Siegel modular form is invariant under the transformation 
T I— > T + 6 and therefore admits a Fourier expansion 

ip = Y^ a{T) exp(27rzTr(TT)), 

T 

where the summation ranges over all 2 x 2 symmetric 'half-integer' matrices, i.e., 
symmetric matrices with integer entries on the diagonal and off-diagonal entries 
in iZ. The coefficients a{T) are called the Fourier coefficients of ij.). As discovered 
by Koecher [20] , they are zero unless T is positive semi-definite. For T = ( ) 
and T = ( ) we have 

Tr (Tr) = ari -|- 6t2 + CT3. 

Writing qi — exp(27rzri), we see that we can express a modular form as t/^ = 
Sfc I m ^k,i,mQiQ2QT ■ Kocchcr's result, the summation ranges over non-negative 
k, I and m satisfying 4m — kl >0. 
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A Siegel modular form is called a cusp form if the Fourier coefficients a(T) are 
zero for all T that are semi-definite but not definite. One of the classical examples 
of a cusp form is 

Xio = -43867 • 2"^^ • 3"^ • 5"^ • 7"^ • 53"^(^4^6 - ^lo), 

which appears in the denominator of the Igusa j-functions. If we express xio in its 
'gj-expansion', every term is divisible by qiq2(lz- The 'normalization factor' ensures 
that the q'iq'2g3-term has coefficient 1. 

Lemma 5.1. The Igusa functions ji have a Laurent series expansion in qi, q2, qs 

with rational coefficients. 

Proof. The denominator of all three Igusa functions is a constant multiple of a 
power of the cusp form xio- The product {qiq2Q3)~^Xio has a non-zero constant 
term and is therefore invertible in the ring C[[gi, 52, qs]]- This shows that the Igusa 
functions have a Laurent series expansion. 

As the Fourier coefficients of the Eisenstein series are rational [11, Corollary 2 
to Theorem 6.3], the coefficients of the Laurent expansion of ji are rational. □ 

In genus 1, it is not hard to prove that the Fourier coefficients of the j-function are 
rational. A deeper result is that they are integral. This is no longer true in genus 2: 
the coefficients of the expansion of the Igusa functions have 'true' denominators. 

Theorem 5.2. For any prime p, the modular polynomials Pp., Qp, Rp lie in the 
ring Qijij2j3)[X]. 

Proof. We only give the proof for Pp, the proof for Qp and Rp is highly similar. 
We can write ^ 

J-, Ea,6.c '^"i,a,6,cJl J2J3 ^rn /r i\ 

^>0 2^a,b,c'^Tn,a,b,c3lJ2J3 

and we have to prove that the coefficients Cm,a,b,c and dm,a,b,c are rational. We 
substitute the Laurent series expansion of Ji, J2, Js, Ji.p into the equation Pp = 0. 
By equating powers of gig2?35 S^^ a set of linear equations for the Cm,a,b,c and 

dm,a,b,c- 

Over the complex numbers, this system of equations has a unique solution. As 
the coefficients of the equations are rational by Lemma 5.1, this solution must be 
rational. □ 

Remark 5.3. We can reduce the polynomials Pp,Qp, Rp modulo a prime I. A 
natural question is if these reduced polynomials still satisfy a moduli interpretation 
as in (4.2). Whereas reduction of modular curves is relatively well understood, the 
situation is more complicated for general Siegel modular varieties. In our situation. 
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the answer is given by a theorem of Chai and Norman [7, Corollary 6.1.1]. They look 
at the algebraic stack A„ ^,(2), . and prove that the structural morphism ^.(2), . 
Spec Z is faithfully flat, Cohen-Macaulay and smooth outside p. Concretely, this 
means that the moduli interpretation (4.2) remains valid modulo primes I ^ p. 

6. Explicit computations 

In this Section we give a method to compute the modular polynomials Pp,Qp,Rp e 
Q(ii,i2,i3) and indicate what the computational difficulties are. We begin with 
the degree of Pp. 



Lemma 6.1. For a prime p, we have [Sp(4, Z) : T^^(p)] = (p^ — l)/(p — 1). 

Proof. By Lemma 3.1, we have to count the number of 2-dimensional isotropic 
subspaces of the symplectic space V = Fp. 

Any two-dimensional isotropic subspacc of V contains (p^ — l)/{p — 1) = p + 1 
lines. Conversely, a line Z C is contained in (p + 1) isotropic subspaces. To see this, 
we note that we can need to select a second line m such that (/, m) is isotropic of 
dimension 2. The complement of I is 3-dimensional, and out of the (p^ — l)/{p—l) 
lines, only (jp — l)/(p — 1) = p + 1 yield an isotropic subspace. 

We see that the number of 2-dimensional isotropic subspaces equals the number 
of lines in V. This yields the lemma. □ 

(2) 

Remark. It is easy to give coset representatives for S = Sp(4, Z)/rQ (p). In 
genus 1, we can take the set 

1 0\ , . „ 1 r ^0 -1 

i 1 



i^F, U I 1 (6.1) 



as a set of coset representatives for SL(2, Z)/Tq{p). Inspired by the set in (6.1) we 
write down 

|a,6,ceFp|u|(^°^ ~J^S^^ \ac=b^eFpY (6.2) 
a set of cardinality p^ +p'^. We are missing p+1 matrices. One can check that 

r„ v; iioeF^u-ii "„„" 1^ (6.3) 

-o 1 

is a set of p + 1 matrices that is independent of the set in (6.2). We note that this 
is the same set of matrices that Dupont found in his thesis [10]. 

To compute Rp and Qp, we have to write j2,p and js^p as rational functions in 
Ji) J2, is and ji^p. For M ranging over the cosets S = Sp(4, Z)/ro (p), the functions 
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ji^p{Mr) are distinct by Lemma 4.2. Inspired by the formulas in [16, Section 7.1] 
we note that, by Lagrange interpolation, the polynomials 



satisfy -Ffc,p(ii,p(Cr)) = j2,p{CT) for = 2,3 and all C G S". As the coefficients of 
Fk^p are, by construction, invariant under the action of Sp(4, Z) the polynomials 
Fk,p are contained in Q(ji, j2, Ja) [-'^l- We have Rp = F2,p{ji,p) and Qp = F3,p(ji,p). 
We have Pp = YImeS^-^ ~ h,p{MT)) and with 



we have Rp = F2,p{ji,p)/Pp{ji,p) and Qp = -F3,p(ii,p)/^^p(ii,p)- Here, denotes the 
derivative of Pp. We deduce that it suffices to compute the 3 polynomials Pp, F2,p 
and Fs^p. In Lemma 6.2 below we prove that the denominators of the coefficients 
of these polynomials are closely related to {p,p)-split Jacobians. 

We say that a principally polarized abelian variety A/C is (p, p)-split if there 
exists an isogeny of degree between A and the product E x E' of two elliptic 
curves with the product polarization. The locus of such A is denoted by Cp. It is 
well known that Cp is a 2-dimensional algebraic subvariety of the 3-dimensional 
moduli space A2- We have chosen coordinates ji,j2, j3 for A2, and Cp can be given 
by an equation Lp = for a polynomial Lp e Q[ji, j2, js]- 

Lemma 6.2. The denominators of the coefficients of Pp, F2^p and F^ p are all 
divisible by the polynomial Lp describing the moduli space of {p,p)-split Jacobians. 

Proof. Let r G H2 correspond to a (p, p)-split Jacobian, and let c be a coefficient 

(2) 

of Pp. For some M G Sp(4, Z)/rQ '{p), the value ji^p{MT) is infinite because the 
functions ji have poles at products of elliptic curves. The evaluation of c at r is a 
symmetric expression in the ji^p(Mt)'s. Generically, there is no algebraic relation 
between these values, and the evaluation of c at r is therefore infinite. 

Since jiir) is finite, the numerator of c is finite. We conclude that the denomi- 
nator of c must vanish at r, i.e., c is divisible by Lp. The proof for F2,p and Fs,p 
proceeds similarly. □ 

Remark. Points on the variety Cp are in bijection with points on the Humbert 
surface Hp2, see [23]. It is a traditionally hard problem to compute equations for 
Humbert surfaces. Up to now, this has only been done for p — 2,3, 5, see [26], [27] 
and [21]. As computing modular polynomials is an even harder problem, it seems 
unlikely that we will be able to compute many examples in the near future. More 
precisely, we conjecture that the following is true. 




E n ^-jhpiBr) b2,p(MT) G Clih,j2,j3)[X] 
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Conjecture 6.3. There exists an element c e Q>o with the property that the 
polynomial Pp requires at least cp^^ hits to write down. 

Heuristic reason. The degree of Pp equals (p^ — l)/{p—l)^p^ by Lemma 6.1. 
The numerator of every coefficient of Pp is a polynomial in ji, j2,i3- We expect 
that the degree in each of the three variables Ji,j3,j3 of these polynomials is of 
order p. Furthermore, by looking at the 1-dimensional case it seems reasonable to 
assume that these polynomials themselves have coefficients of size at least p. This 
means that we need at least cp^ bits to store one coefficient of Pp for some constant 



In the case p = 2 it is relatively straightforward to compute the polynomial L2 
describing (2, 2) -split Jacobians. We refer to [26] for its construction. We have 



L2 = 236196if - 972j4j2 + 5832jfj2j3 + 19245600j4j2 - 8748^4^32 

-104976000ifi3 + 125971200000^4+^3^-4 _ ujfjU^ - 77A36jfjl 
+5^j!jhi + 870912jfjijs - 507384000^3^1 - 108^2^2^33 - 3090960^3^2^1 
+2099520000^3^2^3 + Slj^jl + 3499200^3^33 + JSjfjl - lS32j^jlh 
+592272jfjl + 8910j!jljl - A7m60j^jlh - 29376jf j^ji + 9331200j?j|j| 
+A7952jf hJl - 31104j2j| - 159jij| + lll^jijlh - 41472jij| 
-6048jij|j| + 6912jij3i3 + ml - 384j|j3. 



In the remainder in this section we describe the explicit computation of the en- 
tire polynomial P2- Our idea is to use an interpolation technique, i.e., compute 
-P2(ii('7'), J2(t), j3(r)) e C[X] for sufficiently many r G H2 and use that informa- 
tion to reconstruct the coefficients of P2- Unfortunately, we need to know the full 
denominators of the coefficients of P2 for this approach to work. The interpola- 
tion problem was also considered by Dupont in his thesis [10]. Without knowing 
Lemma 6.2, he succeeded in computing P2. The approach outlined below is inspired 
by Dupont's ideas. 

Let c(ji, j2, ^3) be a coefficient of P2- The first step is to compute the degree in 
ji:j2,j3 of its numerator n(c) and its denominator d{c). We fix y, z ^ Q(i) and for 
a collection of values Xk & Q(*) we compute a value Tk e H2 with 



by first using Mestre's algorithm [22] to find a genus 2 curve C whose Igusa in- 
variants are {xk,y,z) and then finding Tk G H2 corresponding to Jac(C). We can 
therefore evaluate the univariate rational function c{xk-,y-, z). It is now an easy 
matter to determine the degree in ji of n(c) and d{c). Indeed, we check for which 
values of m, n the matrix 



c e Q>o. 



□ 



Ui{rk),j2{rk),j3{rk)) = {xk,y,z) 



n 



,m 



c{xi,y,z) 



c{x-i_,y, z)x'^ 



\ 



M{m, n) 



\ 



1 



m+n+2 



c{Xm+n+2,y,z) 



c{Xm+n+2,y,z)x: 



,n 



m+n+2 
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has non-zero solution-space for arbitrary Xk G Q(i). The smallest m,n for which 
this is the case are the degrees of n(c) and d{c). We find for instance that the 
constant term of P2 has a numerator of degree 60 and a denominator of degree 51 
in ji. Likewise, we can find the degrees in j2 and J3. The degree in j2 of the 
denominators is 42 for all coefficients and we find 30 for the degree in J3. 

As L2 has degree 7 in j2 and degree 5 in J3, we guess that L2 divides d{c). We 
are still missing a polynomial in ji of degree > 1 for the denominator. To find this 
polynomial, a natural idea is to try with a the difference between the degree 
of d{c) and 6 ■ 5 = 30. One heuristic reason for this is the following: if r G H2 
corresponds to the product of elliptic curves, then the numerator of c is infinite 
at T. Combined with the vanishing of L2 at r, this would mean that c has a pole 
of very high order at such r. To 'compensate' for this, we multiply by jf. One 
can verify that the denominator is indeed by taking x,y, z & Z[i] and looking 
at the denominator of c(x, y, z) G Q{i)- 

Having computed the denominator of c, it is an easy matter to compute the nu- 
merator. Indeed, we can evaluate d{c)c at any point r G H2 and apply interpolation 
techniques to find n(c). As the degrees in ji, j2 and of n(c) are relatively large, 
this does take a large amount of time. The constant term of P2 contains 16795 
monomials for instance, with coefficients up to 200 decimal digits. It takes more 
than 50 megabytes to store P2, Q2, R2- 

Combined with Conjecture 6.3, the fact that P21Q21R2 require more than 50 
megabytes to store means that the computation of P^^Q^^R^ will be a major ef- 
fort. Wc can guess the cost of the computation by combining the asymptotic growth 
predicted by Conjecture 6.3 with the 'initial data' provided by P2- A quick com- 
putation yields that we expect that every coefficient of the numerator of P2 has 
roughly 56000 monomials with coefficients up to 300 decimal digits. As P3 has de- 
gree (3^ — l)/(3 — 1) = 40, this means that we would roughly need 0.7 gigabytes 
to store the numerator of P3. The denominator is fairly small, so we would need 
about 2 gigabytes of memory to store -P3, (53, -R3. This is still feasible with currect 
computers, but we did not attempt the computation. 

7. RiCHELOT ISOGENY 

In this Section we zoom in on the special case p = 2, and explain the link between 

our modular polynomials P2, (52, R2 and the classical 'Richelot isogcny'. The Rich- 
elot isogeny is typically used as a tool for computing the Mordell-Weil group of a 
Jacobian of a genus 2 curve, and we first explain the construction. 
Fix a non-singular curve C/C of genus 2, and pick an equation 

= f{X) 

for C, where / G C[X] is monic of degree 6. We pick a factorization / = ABC of / 
into three monic polynomials, each of degree 2. Writing [A, B] = A'B — AB', with 
A' the derivative of ^, we define the curve C by the equation 



AY^ = [A,B][A,C][B,C]. 



(7.1) 
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Here, A is the determinant of A, B, C with respect to the basis 1, X, X^. A simple 
check shows that the right hand side is again a monic polynomial of degree 6. It is 
separable if and only if A is non-zero. 

Lemma 7.1. The Jacobians of the curves C and C defined above are (2,2)- 

isogenous. Furthermore, every (2, 2)-isogeny from Jac(C) into some principally po- 
larized abelian variety A arises via this construction. 

Proof. The fact that Jac(C) and Jac(C") are (2, 2)-isogenous can be found in [6]. 
To show that every (2, 2)-isogeny is a 'Richelot isogeny', we look at the generic 
case that Jac(C) is not (2, 2)-split. It then suffices to prove that there are [Sp4(Z) : 

Fq (2)] = 15 different equations for the curve C. This is simple combinatorics: we 
have a priori 6 • 5/2 = 15 choices for the polynomial A, and then 6 choices for the 
polynomial B. As the right hand side of (7.1) is invariant under a permutation of 
A, B,C, we get 15 different equations. □ 

The connection with the modular polynomial P2 defined in this paper is as follows. 
Assume that Jac(C) is not (2, 2)-split. The discriminant A is then non-zero for every 
choice of factorization / = ABC. We let be the map sending an Igusa invariant ji 
to ji(Jac(C)) G C. Lemma 7.1 tells us that the 15 roots of (/?(P2) G C[X] are exactly 
the first Igusa invariants of the curves C in (7.1). There are similar relations for 
<y?(i?2) and (fi{Q2)- 
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